The use of corpora for automatic evaluation of grammar inference systems
نویسندگان
چکیده
The evaluation of grammar inference systems is clearly a non-trivial task, as it is possible to have more than one correct grammar for a given language. The ‘looks good to me’ approach, carried out by computational linguists analysing their own grammar inference system results, has prevailed for many years. This paper explores why this method has been so popular, in terms of its strengths, and also why it is no longer adequate as a reliable means to measuring performance. Corpus based methods, that can be performed automatically, are investigated to see how they can meet the needs of this difficult problem.
منابع مشابه
The use of corpora for automatic evaluation of grammar inference systems School of Computing University of Leeds Leeds LS 2 9 JT United Kingdom
The evaluation of grammar inference systems is clearly a non-trivial task, as it is possible to have more than one correct grammar for a given language. The 'looks good to me' approach, carried out by computational linguists analysing their own grammar inference system results, has prevailed for many years. This paper explores why this method has been so popular, in terms of its strengths, and ...
متن کاملA Multilingual Parallel Parsed Corpus as Gold Standard for Grammatical Inference Evaluation
In this article we investigate how (computational) grammar inference systems are evaluated and how the evaluation procedure can be improved. First, we describe the currently used evaluation methods and look at the advantages and disadvantages of each method. The main problems of the methods are: the dependency on language experts, the influence of the annotation scheme of language data, and the...
متن کاملCore Units of Spoken Grammar in Global ELT Textbooks
Materials evaluation studies have constantly demonstrated that there is no one fixed procedure for conducting textbook evaluation studies. Instead, the criteria must be selected according to the needs and objectives of the context in which evaluation takes place. The speaking skill as part of the communicative competence has been emphasized as an important objective in language teaching. The pr...
متن کاملAutomatic Detection of Syllable Boundaries Combining the Advantages of Treebank and Bracketed Corpora Training
An approach to automatic detection of syllable boundaries is presented. We demonstrate the use of several manually constructed grammars trained with a novel algorithm combining the advantages of treebank and bracketed corpora training. We investigate the effect of the training corpus size on the performance of our system. The evaluation shows that a hand-written grammar performs better on findi...
متن کاملImprovement of the Reliability of Automatic Manufacture Systems by Using FTA Technique
In recent years, Many manufacturing industries for promoting their efficiency have tended to use the automatic manufacturing systems. Expanding automatic systems and to increase their complexity are representing the necessity of studying a proper functional quality and using reliable equipment in such systems more than ever. In this direction, the technique of fault tree analysis (FTA), along w...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2003